A New Layout Method for Graphical User Interfaces

International audienceThe layout mechanisms for many GUI toolkits are hard to understand, the associated tools and API's often difficult to use. This work investigates new, easy-to-understand layout mechanisms and evaluates its implementation. We will analyze the requirements for the definition of layouts of a graphical user interface. Part of the issue is that several aspects need to be considered simultaneously while laying-out a component: the alignment with other components as well as its own behaviour while resizing its container. Moreover, the used tools should isolate the designer/drawer from the implementation details of the framework. We present the details of our new GUI layout system, discuss the choices we made for our new layout algorithm and detail implementation issues. Moreover, we present also the user interface for our new GUI builder system that contains several innovations, such as a preview window to show the effects of layout configuration choices in real-time. We present an evaluation of our new system by attacking the complex GUI layout problem mentioned above

Reverseorc:Reverse engineering of resizable user interface layouts with or-constraints

Reverse engineering (RE) of user interfaces (UIs) plays an important role in software evolution. However, the large diversity of UI technologies and the need for UIs to be resizable make this challenging. We propose ReverseORC, a novel RE approach able to discover diverse layout types and their dynamic resizing behaviours independently of their implementation, and to specify them by using OR constraints. Unlike previous RE approaches, ReverseORC infers flexible layout constraint specifications by sampling UIs at different sizes and analyzing the differences between them. It can create specifications that replicate even some non-standard layout managers with complex dynamic layout behaviours. We demonstrate that ReverseORC works across different platforms with very different layout approaches, e.g., for GUIs as well as for the Web. Furthermore, it can be used to detect and fix problems in legacy UIs, extend UIs with enhanced layout behaviours, and support the creation of flexible UI layouts.Comment: CHI2021 Full Pape

ORCSolver: An Efficient Solver for Adaptive GUI Layout with OR-Constraints

OR-constrained (ORC) graphical user interface layouts unify conventional constraint-based layouts with flow layouts, which enables the definition of flexible layouts that adapt to screens with different sizes, orientations, or aspect ratios with only a single layout specification. Unfortunately, solving ORC layouts with current solvers is time-consuming and the needed time increases exponentially with the number of widgets and constraints. To address this challenge, we propose ORCSolver, a novel solving technique for adaptive ORC layouts, based on a branch-and-bound approach with heuristic preprocessing. We demonstrate that ORCSolver simplifies ORC specifications at runtime and our approach can solve ORC layout specifications efficiently at near-interactive rates.Comment: Published at CHI202

ORC Layout: Adaptive GUI Layout with OR-Constraints

We propose a novel approach for constraint-based graphical user interface (GUI) layout based on OR-constraints (ORC) in standard soft/hard linear constraint systems. ORC layout unifies grid layout and flow layout, supporting both their features as well as cases where grid and flow layouts individually fail. We describe ORC design patterns that enable designers to safely create flexible layouts that work across different screen sizes and orientations. We also present the ORC Editor, a GUI editor that enables designers to apply ORC in a safe and effective manner, mixing grid, flow and new ORC layout features as appropriate. We demonstrate that our prototype can adapt layouts to screens with different aspect ratios with only a single layout specification, easing the burden of GUI maintenance. Finally, we show that ORC specifications can be modified interactively and solved efficiently at runtime

Visual Analytics: A Method to Explore Natural Histories of Oral Epithelial Dysplasia

Risk assessment and follow-up of oral potentially malignant disorders in patients with mild or moderate oral epithelial dysplasia is an ongoing challenge for improved oral cancer prevention. Part of the challenge is a lack of understanding of how observable features of such dysplasia, gathered as data by clinicians during follow-up, relate to underlying biological processes driving progression. Current research is at an exploratory phase where the precise questions to ask are not known. While traditional statistical and the newer machine learning and artificial intelligence methods are effective in well-defined problem spaces with large datasets, these are not the circumstances we face currently. We argue that the field is in need of exploratory methods that can better integrate clinical and scientific knowledge into analysis to iteratively generate viable hypotheses. In this perspective, we propose that visual analytics presents a set of methods well-suited to these needs. We illustrate how visual analytics excels at generating viable research hypotheses by describing our experiences using visual analytics to explore temporal shifts in the clinical presentation of epithelial dysplasia. Visual analytics complements existing methods and fulfills a critical and at-present neglected need in the formative stages of inquiry we are facing

WiseType : a tablet keyboard with color-coded visualization and various editing options for error correction

To address the problem of improving text entry accuracy in mobile devices, we present a new tablet keyboard that offers both immediate and delayed feedback on language quality through auto-correction, prediction, and grammar checking. We combine different visual representations for grammar and spelling errors, accepted predictions, and auto-corrections, and also support interactive swiping/tapping features and improved interaction with previous errors, predictions, and auto-corrections. Additionally, we added smart error correction features to the system to decrease the overhead of correcting errors and to decrease the number of operations. We designed our new input method with an iterative user-centered approach through multiple pilots. We conducted a lab-based study with a refined experimental methodology and found that WiseType outperforms a standard keyboard in terms of text entry speed and error rate. The study shows that color-coded text background highlighting and underlining of potential mistakes in combination with fast correction methods can improve both writing speed and accuracy

How Automatic Speed Control Based on Distance Affects User Behaviours in Telepresence Robot Navigation Within Dense Conference-like Environments

Telepresence robots allow users to be spatially and socially present in remote environments. Yet, it can be challenging to remotely operate telepresence robots, especially in dense environments such as academic conferences or workplaces. In this paper, we primarily focus on the effect that a speed control method, which automatically slows the telepresence robot down when getting closer to obstacles, has on user behaviors. In our first user study, participants drove the robot through a static obstacle course with narrow sections. Results indicate that the automatic speed control method significantly decreases the number of collisions. For the second study we designed a more naturalistic, conference-like experimental environment with tasks that require social interaction, and collected subjective responses from the participants when they were asked to navigate through the environment. While about half of the participants preferred automatic speed control because it allowed for smoother and safer navigation, others did not want to be influenced by an automatic mechanism. Overall, the results suggest that automatic speed control simplifies the user interface for telepresence robots in static dense environments, but should be considered as optionally available, especially in situations involving social interactions

REAL-TIME COMPUTATION OF AREA SHADOWS- A GEOMETRICAL ALGORITHM

The computation of soft shadows created by area light sources is a well-known problem in computer graphics. Due to the complexity of the problem, soft shadows commonly are generated only for images that are rendered in an off-line process. In interactive virtual environments, where images have to be computed in real time, soft shadows are mostly replaced by hard shadows as this takes much less time to compute. This paper presents an algorithm that uses a geometrical method to generate a triangulated approximation of the soft shadow cast by a polygonal object very quickly. The algorithm can simulate effects such as the varying width of the penumbra depending on distance to the light and can be extended to support non-convex light sources as well. We mention the artifacts arising from triangulation and give discuss how to alleviate such problems